Sensor management for wireless devices

ABSTRACT

A system and method for selecting audio capture sensors of wearable devices in obtaining voice data. The method provides obtaining signals associated with the user&#39;s voice at first and second wearable devices, comparing energy levels of the first and second signals, and selecting one or more audio capture sensors based on the energy levels of each signal. Due to the symmetry of the acoustic energy produced by the user&#39;s voice to a first and second wearable device, any difference in energy level between the total energy obtained by the first wearable device and the total energy obtained by the second wearable device can be attributed solely to ambient noise. Thus, the device with the higher total energy has a lower signal-to-noise ratio and selection of an audio capture sensor of the other wearable device with a higher signal-to-noise ratio is provided to obtain voice data moving forward.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application and claims benefit ofU.S. patent application Ser. No. 17/126,630, filed Dec. 18, 2020, thecontents of which are herein incorporated by reference in theirentirety.

BACKGROUND

Aspects and implementations of the present disclosure are generallydirected to systems and methods for managing sensor data, for example,managing the capture of voice data from one or more audio capturesensors between wearable devices.

Wearable wireless audio devices, e.g., wireless headphones, oftenutilize paired connections to stream wireless audio data from a sourcedevice. Typically, each wireless headphone receives a discrete stream ofdata specific to each wireless headphone, e.g., the source deviceproduces one stream of data associated with the left headphone and onestream associated with the right headphone. In some circumstances, eachheadphone may also utilize one or more microphones to obtain a signalcorresponding to a user's voice, e.g., when the user is engaged in atelephone conversation where the headphones (via a Bluetooth connection)act as both the receiver and transmitter for the conversation. In someexamples, only one headphone of the pair of headphones is used to obtainthe signals corresponding to the user's voice.

Additionally, at various times, the environment surrounding the user mayproduce excessive ambient noise cause by, for example, wind, traffic,machinery, etc. If the user is oriented such that the one of theheadphones, e.g., the headphone responsible for obtaining the signalscorresponding to the users voice, is closer to the source of the ambientnoise, e.g., in the direct path of wind, the quality of the voice pickupof that headphone will suffer.

SUMMARY OF THE DISCLOSURE

The present disclosure relates to systems and methods and computerprogram products for selecting one or more audio capture sensors ofwearable devices for use in obtaining voice data. The examples providedinclude obtaining signals associated with the user's voice at a firstand a second wearable device (where the first wearable device ispositioned in the user's right ear and the second wearable device ispositioned in the user's left ear), comparing energy levels of the firstand second signals, and selecting one or more audio capture sensorsbased on the energy levels of each signal. Due to the symmetry of theacoustic energy produced by the user's voice to a first and secondwearable device (as the device in the user's right ear is equallydistant from the device in the user's left ear), any difference inenergy level between the total energy obtained by the first wearabledevice and the total energy obtained by the second wearable device canbe attributed solely to a difference in ambient noise. Thus, the devicewith the higher total energy has a lower signal-to-noise ratio andselection of one or more audio capture sensors of the other wearabledevice with the higher signal-to-noise ratio is preferred to obtainvoice data moving forward.

In some examples, the systems and methods discussed herein utilizewireless data transmission, specifically, wireless topologies forbroadcasting and transmitting audio streams between devices. Forexample, Core Specification 5.2 (“The Core Specification”) released bythe Bluetooth Special Interest Group (SIG) on Jan. 6, 2020, defines newfeatures related to Bluetooth Low Energy (BLE) topologies. One featuredescribed in the 5.2 Core Specification is Broadcast Isochronous Streamswhich utilize connectionless isochronous communications. A similarfeature described by the 5.2 Core Specification is an LE ConnectedIsochronous Stream, which utilizes connection-oriented isochronouschannels to provide a point-to-point isochronous communication streambetween two devices, e.g., between peripheral device 104 and wearabledevices 102A and 102B (discussed below). In one example, the systems,devices, and methods discussed herein utilize Bluetooth Low-Energy audiotopologies enabled by the 5.2 Core Specification (referred to herein as“LE Audio”). For example, LE Audio enables unicast wireless topologies(referred to as “connected isochronous streams”) that allow a singleBluetooth audio source device (e.g., a smart phone) to transmit multipleaudio streams to separate Bluetooth devices at the same time, e.g.,wireless headphones. These topologies are intended to improve Bluetoothoperation for wireless headphones.

Specifically, when using wireless headphone or wireless wearable device(discussed below) algorithms can be used to improve voice pickup usinginputs from multiple microphones on wearable devices. Some of thesealgorithms minimize ambient noise, e.g., wind noise, by reducing gain onmicrophones which are directly in the path of wind. Other algorithms usebinaural beamforming (using microphones placed close to both left andright ears) to maximize signal-to-noise ratio. Many of these algorithmscannot be realized on truly wireless earbud form factors because of thehigh latency between the earbuds. These truly wireless earbuds generallyuse Bluetooth to transmit data between earbuds and it can take up to 100ms to transfer microphone data reliably between earbuds. This exceedsthe acceptable latency that some voice pickup algorithms can tolerate.

In this disclosure, sensor inputs (e.g., microphone inputs) aremonitored on both left and right wearable devices to determine which setof sensors provides an optimal signal-to-noise ratio. In some exampleswhich use classic Bluetooth connections, the “master” earbud (whichmaintains the Bluetooth connection to the peripheral device) is thedevice that is configured to send microphone audio data to theperipheral device (e.g., a smart phone) during a voice interaction. If,through a communication data link between the two earbuds, it isdetermined that ambient noise, such as wind noise, is high in the masterbud but low in the puppet bud, the two buds can initiate a role switchmid-stream. In some examples, the systems and method utilize one or moreclassic Bluetooth chip solutions which enabling “seamless” roleswitching on a classic Bluetooth connection that could enable this torole switch. This allows the master bud to hand off responsibility formaintaining the Bluetooth connection to the smartphone to the puppetbud, enabling the puppet bud to use its microphones (with lower windnoise) for voice pickup instead of the master bud's microphones. To doso without a role switch would require the puppet's microphone audio tobe sent over Bluetooth to the master bud and then forwarded to thephone, which would produce unacceptably high latency for real time voiceconversations.

A similar scheme can be employed for LE audio connections. In LE audio,each bud may be synchronized to a separate Connected Isochronous Stream(CIS) which are part of the same Connected Isochronous Group (CIG). Eachbud could have knowledge of the other bud's CIS either by monitoringconnection setup of the CIS for the other bud or by exchanginginformation via a communication data link between buds. When used for aphone call, for instance, one of the two buds may transmit microphoneaudio to a negotiated CIS sent to the phone. Upon detecting significantambient noise, e.g., wind noise, on that microphone audio stream, thebud could do a handshake with the other bud and “hand off”responsibility for sending microphone audio to the other bud. This wouldallow the second bud to send its microphone data to the phone (with lesswind noise) without the phone knowing that a switch had been made.

In both classic Bluetooth and LE audio implementations, automatic gaincontrol parameters for each microphone could be exchanged over thecommunication data link between the buds before making the role switchbetween microphones, to prevent users from noticing a difference inmicrophone gain when microphones are switched.

LE audio capability could be leveraged in additional ways to improvevoice pickup for truly wireless buds. In one implementation, raw(unprocessed) audio data from one or two microphones could be sent fromeach bud to a receiving device (like a phone). The phone could receiveaudio from all microphones and then apply beamforming or other voicepickup algorithms on the raw audio data. This could be done in one ormore applications or could utilize a voice algorithm “plugin” to other3rd party VOIP apps. In other implementations, one or more of thesealgorithms could be integrated into VPA apps or be integrated into cloudservices. The algorithms could be tuned optimally for different devicesand could be selected by a mobile app or cloud services based onidentifying information from the device. Performing voice processing onthe phone or cloud side enables multi-mic voice algorithms to bedeployed using microphones from both sides of the head, which is muchmore difficult to accomplish for truly wireless buds due to delays inaudio transmission from bud to bud.

In other implementations, to save Bluetooth bandwidth, each bud could dolocal beamforming of two microphones and only send a single channel ofvoice audio to the phone. The phone or cloud can then run algorithms (ina similar manner mentioned above) to processed beamformed audio fromeach bud and combine them to produce improved voicepickup/signal-to-noise ration of the audio signal.

In another implementation, an active noise reduction (ANR) feedbackmicrophone (facing towards the user's ear) could be used to capturevoice from one earbud in windy conditions, while on the other bud anexternally facing microphone could be used for voice capture. The ANRfeedback microphone, while more immune to wind noise (or other highnoise environments like babble noise), is not able to pick up a lot ofhigh frequency content. The externally facing microphone in the otherbud, when mixed with voice audio from the ANR feedback microphone fromthe first earbud, enables the bandwidth of the voice capture to beextended while minimizing wind noise. As wind conditions changedynamically, the buds could coordinate swapping which bud uses an ANRfeedback microphone to capture voice and which one uses an externallyfacing microphone.

Both buds could also capture voice using ANR feedback and externallyfacing microphones simultaneously and compress the microphone audio andsend multiple microphone channels to the source device, where mixingcould be done. Algorithms running on the source device could thendynamically select which microphones are used for mixing, based on theenvironmental noise picked up by externally facing microphones in eachbud. Any voice algorithm running independently on each side can alsosend key information (e.g., through metadata) to the source device whichthen uses that information to decide which microphones to utilize formixing (mics from one bud or another or a mix of both). Non-standardcodecs could also be used for mic audio transmission that have lowerencoding/decoding algorithmic delay or higher compression as appropriatefor a given use case.

In other use cases, the output of the aforementioned microphone mixingscheme can be used to make decisions regarding the volume level or ANRsettings of the product. For instance, based on the noise levelsdetected by the microphone mixing algorithms running on the sourcedevice, a signal could be sent back to the product telling it to adjustthe volume level to compensate for environmental noise. Similarly,information about environmental noise could be sent as a signal back tothe product in order to automatically adjust ANR levels.

In one example, a method for selecting one or more audio capture sensorsof wearable devices is provided, the method including: detecting a firstsignal corresponding with a user's voice using at least one audiocapture sensor of a first wearable device; detecting a second signalcorresponding with the user's voice using at least one audio capturesensor of a second wearable device, wherein the second wearable deviceis wirelessly connected to the first wearable device; determining anenergy level of the first signal; determining an energy level of thesecond signal; and selecting, based at least in part on the energy levelof the first signal and the energy level of the second signal, at leastone audio capture sensor of the first wearable device and/or at leastone audio capture sensor of the second wearable device to obtain voicedata.

In one aspect, the first wearable device is configured to send the firstsignal to a peripheral device and the second wearable device isconfigured to send the second signal to a peripheral device, and whereinthe peripheral device is configured to compare the energy level of thefirst signal with the energy level of the second signal for theselecting of the at least one audio capture sensor of the first wearabledevice and/or the at least one audio capture sensor of the secondwearable device to obtain voice data.

In one aspect, the at least one audio capture sensor of the firstwearable device comprises a first set of multiple audio capture sensors,and wherein the first signal is an average energy level of the firstmultiple audio capture sensors, and wherein the first wearable device isconfigured to send the first signal to the peripheral device using acompression algorithm.

In one aspect, the first wearable device is further configured to sendmetadata to the first peripheral device for use with a mixing algorithmexecuted by the peripheral device.

In one aspect, the mixing algorithm is arranged to generate an output,and wherein the peripheral device is configured to send the output tothe first wearable device, wherein the first wearable audio device isconfigured to use the output to automatically change a volume level oran active noise reduction (ANR) setting.

In one aspect, the second wearable device is configured to send thesecond signal to the first wearable device, and wherein the firstwearable device is configured to compare the energy level of the firstsignal with the energy level of the second signal for the selecting ofthe at least one audio capture sensor of the first wearable deviceand/or the at least one audio capture sensor of the second wearabledevice to obtain voice data.

In one aspect, the energy levels of the first and second signals relateto at least one of signal-to-noise ratio (SNR), wind presence, radiofrequency (RF) performance, or audio pickup.

In one aspect, the at least one audio capture sensor of the firstwearable device and the at least one audio capture sensor of the secondwearable device are selected from at least one of an exteriormicrophone, an interior microphone, a feedback microphone that is alsoused for acoustic noise reduction (ANR) purposes, a feedforwardmicrophone that is also used for ANR purposes, or an accelerometer.

In one aspect, the at least one audio capture sensor of the firstwearable device includes a first set of multiple audio capture sensors,and wherein the selecting of the at least one audio capture sensorincludes dynamically changing between selection of a first audio capturesensor of the first set of multiple audio capture sensors and a secondaudio capture sensor of the first set of multiple audio capture sensors.

In one aspect, the at least one audio capture sensor of the firstwearable device includes a first set of multiple audio capture sensors,and wherein the energy level of the first signal is an average energylevel of the first set of multiple audio capture sensors.

In one aspect, the at least one audio capture sensor of the secondwearable device includes a second set of multiple audio capture sensors,and wherein the energy level of the second signal is an average energylevel of the second set of multiple audio capture sensors.

In one aspect, when the energy level of the first signal is less thanthe energy level of the second signal, only the at least one audiocapture sensor of the first wearable device is selected to obtain voicedata and not the at least one audio capture sensor of the secondwearable device.

In one aspect the voice data is sent to a peripheral device using anegotiated connected isochronous stream.

In one aspect, when the selecting causes a change from using the atleast one audio capture sensor of the first wearable device to obtainvoice data to using the at least one audio capture sensor of the secondwearable device to obtain voice data, the first wearable device performsa handshake with the second wearable device to shift responsibility toobtain voice data to the at least one audio capture sensor of the secondwearable device.

In one aspect, when the selecting causes a change from using the atleast one audio capture sensor of the first wearable device to obtainvoice data to using the at least one audio capture sensor of the secondwearable device to obtain voice data, a gain parameter associated withthe at least one audio capture sensor of the first wearable device isshared with the second wearable device to be applied to the at least oneaudio capture sensor of the second wearable device.

In another example, a computer program product for selecting one or moreaudio capture sensors of wearable devices is provided, the computerprogram product including a set of non-transitory computer readableinstructions that when executed on at least one processor of a firstwearable device, a second wearable device, or a peripheral device, theprocessor is configured to: detect or receive a first signalcorresponding with a user's voice captured by at least one audio capturesensor of the first wearable device; detect or receive a second signalcorresponding with the user's voice captured by at least one audiocapture sensor of the second wearable device, wherein the secondwearable device is wirelessly connected to the first wearable device;determine an energy level of the first signal; determine an energy levelof the second signal; and select, based at least in part on the energylevel of the first signal and the energy level of the second signal, atleast one audio capture sensor of the first wearable device or at leastone audio capture sensor of the second wearable device to obtain voicedata.

In one aspect, the first wearable device is configured to send the firstsignal to the peripheral device and the second wearable device isconfigured to send the second signal to the peripheral device, andwherein the peripheral device is configured to compare the energy levelof the first signal with the energy level of the second signal forselecting of the at least one audio capture sensor of the first wearabledevice or the at least one audio capture sensor of the second wearabledevice to obtain voice data.

In one aspect, the second wearable device is configured to send thesecond signal to the first wearable device, and wherein the firstwearable device is configured to compare the energy level of the firstsignal with the energy level of the second signal for the selecting ofthe at least one audio capture sensor of the first wearable device orthe at least one audio capture sensor of the second wearable device toobtain voice data.

In one aspect, the energy levels of the first and second signals relatedto at least one of signal-to-noise ratio (SNR), wind presence, radiofrequency (RF) performance, or audio pickup.

In one aspect, the at least one audio capture sensor of the firstwearable device and the at least one audio capture sensor of the secondwearable device are selected from at least one of: an exteriormicrophone, an interior microphone, a feedback microphone that is alsoused for acoustic noise reduction (ANR) purposes, a feedforwardmicrophone that is also used for ANR purposes, or an accelerometer.

In one aspect, the at least one audio capture sensor of the firstwireless worn device includes a first set of multiple audio capturesensors and wherein the energy level of the first signal is an averageenergy level of the first set of multiple audio capture sensors; and theat least one audio capture sensor of the second wearable device includesa second set of multiple audio capture sensors, and wherein the energylevel of the second signal is an average energy level of the second setof audio capture sensors.

In one aspect, when the energy level of the first signal is less thanthe energy level of the second signal, only the at least one audiocapture sensor of the first wearable device is selected to obtain voicedata and not the at least one audio capture sensor of the secondwearable device.

In one aspect, the selecting causes a change from using the at least oneaudio capture sensor of the first wearable device to obtain voice datato using the at least one audio capture sensor of the second wearabledevice to obtain voice data, the first wearable device performs ahandshake with the second wearable device to shift responsibility toobtain voice data to the at least one audio capture sensor of the secondwearable device.

In one aspect, the selecting causes a change from using the at least oneaudio capture sensor of the first wearable device to obtain voice datato using the at least one audio capture sensor of the second wearabledevice to obtain voice data, a gain parameter associated with the atleast one audio capture sensor of the first wearable device is sharedwith the second wearable device to be applied to the at least one audiocapture sensor of the second wearable device.

These and other aspects of the various embodiments will be apparent fromand elucidated with reference to the embodiment(s) describedhereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, like reference characters generally refer to the sameparts throughout the different views. Also, the drawings are notnecessarily to scale, emphasis instead generally being placed uponillustrating the principles of the various embodiments.

FIG. 1 is a schematic view of a system according to the presentdisclosure.

FIG. 2A is a schematic view of the components of a first wearable deviceaccording to the present disclosure.

FIG. 2B is a schematic view of the components of a second wearabledevice according to the present disclosure.

FIG. 3 is a schematic view of the components of a peripheral deviceaccording to the present disclosure.

FIG. 4A is a schematic top view of a system according to the presentdisclosure.

FIG. 4B is a schematic top view of a system according to the presentdisclosure.

FIG. 5 is a flow chart illustrating the steps of a method according tothe present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

The present disclosure relates to systems and methods and computerprogram products for selecting one or more audio capture sensors ofwearable devices for use in obtaining voice data. The examples providedinclude obtaining signals associated with the user's voice at a firstand a second wearable device (where the first wearable device ispositioned in the user's right ear and the second wearable device ispositioned in the user's left ear), comparing energy levels of the firstand second signals, and selecting one or more audio capture sensorsbased on the energy levels of each signal. Due to the symmetry of theacoustic energy produced by the user's voice to a first and secondwearable device (as the device in the user's right ear is equallydistant from the device in the user's left ear), any difference inenergy level between the total energy obtained by the first wearabledevice and the total energy obtained by the second wearable device canbe attributed solely to a difference in ambient noise. Thus, the devicewith the higher total energy has a lower signal-to-noise ratio andselection of one or more audio capture sensors of the other wearabledevice with the higher signal-to-noise ratio is preferred to obtainvoice data moving forward.

The term “wearable audio device”, as used in this application, inaddition to including its ordinary meaning or its meaning known to thoseskilled in the art, is intended to mean a device that fits around, on,in, or near an ear (including open-ear audio devices worn on the head orshoulders of a user) and that radiates acoustic energy into or towardsthe ear. Wearable audio devices are sometimes referred to as headphones,earphones, earpieces, headsets, earbuds or sport headphones, and can bewired or wireless. A wearable audio device includes an acoustic driverto transduce audio signals to acoustic energy. The acoustic driver canbe housed in an earcup. While some of the figures and descriptionsfollowing can show a single wearable audio device, having a pair ofearcups (each including an acoustic driver) it should be appreciatedthat a wearable audio device can be a single stand-alone unit havingonly one earcup. Each earcup of the wearable audio device can beconnected mechanically to another earcup or headphone, for example by aheadband and/or by leads that conduct audio signals to an acousticdriver in the ear cup or headphone. A wearable audio device can includecomponents for wirelessly receiving audio signals. A wearable audiodevice can include components of an active noise reduction (ANR) system.Wearable audio devices can also include other functionality such as amicrophone so that they can function as a headset. While FIG. 1 shows anexample of an in-the-ear headphone form factor, in other examples thewearable audio device can be an on-ear, around-ear, behind-ear,over-the-ear or near-ear headset, or can be an audio eyeglasses formfactor headset. In some examples, the wearable audio device can be anopen-ear device that includes an acoustic driver to radiate acousticenergy towards the ear while leaving the ear open to its environment andsurroundings.

The term “connected isochronous stream” as used herein, in addition toincluding its ordinary meaning or its meaning known to those skilled inthe art, is intended to refer to an isochronous data stream whichutilizes a preestablished, point-to-point communication link over LEAudio between, e.g., a source device and an audio device or a pluralityof audio devices. In other words, a connected isochronous stream canprovide an isochronous audio stream which utilizes at least oneestablished reliable communication channel and/or at least oneacknowledged communication channel between the source device and anyrespective audio devices.

The term “broadcast isochronous stream” as used herein, in addition toincluding its ordinary meaning or its meaning known to those skilled inthe art, is intended to refer to an isochronous data stream which doesnot require a preestablished communications link to be establishedbetween the source device sending data and the audio device receivingdata and does not require acknowledgements or negative acknowledgementsto be sent or received.

The following description should be read in view of FIGS. 1-4B. FIG. 1is a schematic view of system 100 according to the present disclosure.System 100 includes a plurality of wearable devices, e.g., firstwearable device 102A and second wearable device 102B (collectivelyreferred to herein as “wearable devices 102”) and a peripheral device104. In the examples illustrated, first wearable device 102A and secondwearable device 102B are intended to be a pair of wearable audiodevices, e.g., a pair of truly wireless earbuds, where first wearabledevice 102A and second wearable device 102B are arranged to be securedproximate to or within a user's right and left ears, respectively.However, in some alternative examples, it should be appreciated thatfirst wearable device 102A and second wearable device 102B can beselected from at least one of: hearing aids, speakers, portablespeakers, paired speakers or paired portable speakers. As illustrated,system 100 includes a peripheral device 104, e.g., a smartphone ortablet, configured to establish wireless data connections with wearabledevices 102, which is discussed below in detail.

As illustrated in FIG. 2A first wearable device 102A comprises firstcircuitry 106. First circuitry 106 includes first processor 108 andfirst memory 110 configured to execute and store, respectively, a firstplurality of non-transitory computer-readable instructions 112, toperform the various functions of first wearable device 102A and firstcircuitry 106 as will be described herein. First circuitry 106 alsoincludes a first communications module 114 configured to send and/orreceive wireless data, e.g., data relating to at least one of theplurality of communication data streams discussed below, e.g.,communication data stream 162A. To that end, first communications module114 can include at least one radio or antenna, e.g., a first radio 116capable of sending and receiving wireless data. In some examples, firstcommunications module 114 can include, in addition to at least one radio(e.g., first radio 116), some form of automated gain control (AGC), amodulator and/or demodulator, and potentially a discrete processor forbit-processing that are electrically connected to first processor 108and first memory 110 to aid in sending and/or receiving wireless data.As will be discussed below, first circuitry 106 of first wearable device102A can also include a first speaker 118, e.g., a loudspeaker oracoustic transducer, that is electrically connected to first processor108 and first memory 110 and configured to electromechanically convertan electrical signal into audible acoustic energy within environment E,e.g., an audio playback. In some examples, the electrical signal and theaudible acoustic energy are associated with the data included in theplurality of communication data streams (discussed below).

First wearable device 102A can further include a first set of audiocapture sensors 120. In some examples, the first set of audio capturesensors includes only one audio capture sensor. In some examples, thefirst set of audio capture sensors 120 includes multiple audio capturesensors, e.g., a first audio capture sensor 120A and a second audiocapture sensor 120B (collectively referred to as “first set of audiocapture sensors 120”). In other examples, the first set of audiocaptures sensors 120 can have more than two audio capture sensors, e.g.,the set can include 3, 4, 5, or more audio capture sensors. Each audiocapture sensor of the first set of audio capture sensors 120 is intendedto be a microphone or some other audio capture device (e.g., aunidirectional or directional micro-electro-mechanical system (MEMS)microphone arranged on, in, or in proximity to the first wearable device102A). It should be appreciated that each audio capture sensor 120 canbe configured as: an external microphone (e.g., a microphone positionedto pickup or obtain sound energy outside of the air cavity createdbetween the wearable audio device 102A and the user's eardrum); aninternal microphone (e.g., a microphone positioned to pickup or obtainsound energy inside of the air cavity created between the wearable audiodevice 102A and the user's eardrum); or an accelerometer. In someexamples, the exterior microphone can be a feedforward microphone alsoused to obtain or pickup external sound energy used in active noisereduction (ANR) applications. In other examples, the internal microphonecan be a feedback microphone also used to obtain or pickup internalsound energy used in ANR applications. It should be appreciated thateach audio capture sensor can be configured to obtain or pick up soundenergy via air conduction and/or via bone conduction (through one ormore bones of the user's body, e.g., the user's head or jaw). It shouldalso be appreciated that two or more audio capture sensors of the firstset of audio capture sensors 120 can be arranged such that beamformingtechniques, using one or more algorithms, can be utilized to enhance thequality of the audio pickup obtained by the audio capture sensors, e.g.,to enhance the quality of voice data 164 (discussed below). In someimplementations, multiple of the same type of audio capture sensor areincluded on a wearable device, such as where there are two externalfacing microphones for audio capture sensors 120A and 120B on firstwearable device 102A. In some implementations, a mix of different typesof audio capture sensors are in included on a wearable device, such aswhere first audio capture sensor 120A is an external facing microphoneand second audio capture sensor 120B is an accelerometer.

As will be discussed below, each audio capture sensor of the first setof audio capture sensors 120 is configured to obtain sound energy fromthe environment surrounding the user and convert that sound energy intoan electronic signal, i.e., first signal 122. Once obtained, firstsignal 122 can be analyzed to determine a first energy level 124. Firstenergy level 124 is intended to be a measure of the total acousticenergy detected one or more audio capture sensors 120. This totalacoustic energy can include acoustic energy generated by the user, e.g.,the user's voice, as well as other ambient acoustic energy from thesurrounding environment, e.g., wind, conversations, traffic noise,machinery, etc. In some examples where the first set of audio capturesensors 120 includes two or more audio capture sensors, as describedabove, first energy level 124, can be an average energy level, i.e.,first average energy level 126, obtained or picked up by all of theaudio capture sensors of the first set of audio capture sensors 120. Forexample, one or more external microphone signals may be averagedtogether to form first energy level 124. In another example, one or moreinternal microphone signals may be averaged with one or more externalmicrophones to form first energy level 124.

As illustrated in FIG. 2B, second wearable device 102B comprises secondcircuitry 128. Second circuitry 128 includes second processor 130 andsecond memory 132 configured to execute and store, respectively, asecond plurality of non-transitory computer-readable instructions 134,to perform the various functions of second wearable device 102B andsecond circuitry 128 as will be described herein. Second circuitry 128also includes a second communications module 136 configured to sendand/or receive wireless data, e.g., data relating to the plurality ofdata streams discussed below, e.g., isochronous data stream 162B. Tothat end, second communications module 136 can include at least oneradio or antenna, e.g., a second radio 138 capable of sending andreceiving wireless data. In some examples, second communications module128 can include, in addition to at least one radio (e.g., first radio138), some form of automated gain control (AGC), a modulator and/ordemodulator, and potentially a discrete processor for bit-processingthat are electrically connected to second processor 130 and secondmemory 132 to aid in sending and/or receiving wireless data. As will bediscussed below, second circuitry 128 of second wearable device 102B canalso include a second speaker 140, e.g., a loudspeaker or acoustictransducer, that is electrically connected to second processor 130 andsecond memory 132 and configured to electromechanically convert anelectrical signal into audible acoustic energy within environment E,e.g., an audio playback. In some examples, the electrical signal and theaudible acoustic energy are associated with the data included in theplurality of data streams (discussed below).

Second wearable device 102B can further include a second set of audiocapture sensors 142. In some examples, the second set of audio capturesensors includes only one audio capture sensor. In some examples, thesecond set of audio capture sensors 142 includes multiple audio capturesensors, e.g., a first audio capture sensor 142A and a second audiocapture sensor 142B (collectively referred to as “second set of audiocapture sensors 142”). In other examples, the second set of audiocaptures sensors 142 can have more than two audio capture sensors, e.g.,the set can include 3, 4, 5, or more audio capture sensors. Each audiocapture sensor of the second set of audio capture sensors 142 isintended to be a microphone, or some other audio capture device (e.g., aunidirectional or directional micro-electro-mechanical system (MEMS)microphone arranged on, in, or in proximity to the second wearabledevice 102B). It should be appreciated that each audio capture sensor142 can be configured as: an external microphone (e.g., a microphonepositioned to pickup or obtain sound energy outside of the air cavitycreated between the wearable audio device 102B and the user's eardrum);an internal microphone (e.g., a microphone positioned to pickup orobtain sound energy inside of the air cavity created between thewearable audio device 102B and the user's eardrum); or an accelerometer.In some examples, the exterior microphone can be a feedforwardmicrophone also used to obtain or pickup external sound energy used inactive noise reduction (ANR) applications. In other examples, theinternal microphone can be a feedback microphone also used to obtain orpickup internal sound energy used in ANR applications. It should beappreciated that each audio capture sensor can be configured to obtainor pickup sound energy via air conduction and/or via bone conduction(through one or more bones of the user's body, e.g., the user's head orjaw). It should also be appreciated that two or more audio capturesensors of the second set of audio capture sensors 142 can be arrangedsuch that beamforming techniques, using one or more algorithms, can beutilized to enhance the quality of the audio pickup obtained by theaudio capture sensors, e.g., to enhance the quality of voice data 164(discussed below). In some implementations, multiple of the same type ofaudio capture sensor are included on a wearable device, such as wherethere are two external facing microphones for audio capture sensors 142Aand 142B on second wearable device 102B. In some implementations, a mixof different types of audio capture sensors are in included on awearable device, such as where first audio capture sensor 142A is anexternal facing microphone and second audio capture sensor 142B is anaccelerometer.

As will be discussed below, each audio capture sensor of the second setof audio capture sensors 142 is configured to obtain sound energy fromthe environment surrounding the user and convert that sound energy intoan electronic signal, i.e., second signal 144. Once obtained, secondsignal 144 can be analyzed to determine a second energy level 146.Second energy level 146 is intended to be a measure of the totalacoustic energy detected by at least one audio capture sensor 142. Thistotal acoustic energy can include acoustic energy generated by the user,e.g., the user's voice, as well as other ambient acoustic energy fromthe surrounding environment, e.g., wind, conversations, traffic noise,machinery, etc. In some examples where the second set of audio capturesensors 142 includes two or more audio capture sensors, as describedabove, second energy level 146, can be an average energy level, i.e.,second average energy level 148, obtained or picked up by all of theaudio capture sensors of the second set of audio capture sensors 142.For example, one or more external microphone signals may be averagedtogether to form second energy level 146. In another example, one ormore internal microphone signals may be averaged with one or moreexternal microphones to form second energy level 146.

As illustrated in FIG. 3 , system 100 further includes peripheral device104. Peripheral device 104 is intended to be a wired or wireless devicecapable of sending and/or receiving data related to the plurality ofcommunication data streams 162A-162B to at least one wearable device,e.g., first wearable device 102A and/or second wearable device 102B. Inone example, as illustrated in FIG. 1 , peripheral device 104 is asmartphone capable of sending data from plurality of data streams162A-162B to first wearable device 102A and/or second wearable device102B. Although not illustrated, it should be appreciated that peripheraldevice 104 can also be selected from at least one of: a personalcomputer, a mobile computing device, a tablet, a smart speaker, a smartspeaker system, a smart hub, a smart television, or any other devicecapable of sending or receiving data from plurality of data streams162A-162B (discussed below). In some examples, peripheral device 104 isa remote device that is wirelessly paired with first wearable device102A and/or second wearable device 102B. Accordingly, peripheral device104 can comprise peripheral circuitry 150. Peripheral circuitry 150includes peripheral processor 152 and peripheral memory 154 configuredto execute and store, respectively, a plurality of non-transitorycomputer-readable instructions, e.g., peripheral instructions 156, toperform the various functions of peripheral device 104 and peripheralcircuitry 150 as will be described herein. Peripheral circuitry 150 alsoincludes a peripheral communications module 158 configured to sendand/or receive wireless data, e.g., data relating to the plurality ofdata streams 162A-162B (discussed below) to and from wearable devices102. To that end, peripheral communications module 158 can include atleast one radio or antenna, e.g., a peripheral radio 160 capable ofsending and receiving wireless data. In some examples, peripheralcommunications module 158 can include, in addition to at least one radio(e.g., peripheral radio 160), some form of automated gain control (AGC),a modulator and/or demodulator, and potentially a discrete processor forbit-processing that are electrically connected to peripheral processor152 and peripheral memory 154 to aid in sending and/or receivingwireless data. Additionally, peripheral device 104 can include, withinthe set of non-transitory computer-readable instructions, one or moreapplications, e.g., a mobile application capable of interacting with andcommunicating with each wearable device within the system, i.e., atleast first wearable device 102A and second wearable device 102B.

Each device of system 100, i.e., each wearable device 102 and peripheraldevice 104 may use their respective communication modules to establishcommunication data streams between each device. For example, asillustrated in FIG. 1 , system 100 can be configured to establish afirst communication data stream 162A between first wearable device 102Aand peripheral device 104, establish a second communication data stream162B between second wearable device 102B and peripheral device 104, andestablish a third communication data stream 162C between first wearabledevice 102A and second wearable device 102B. Each communication datastream, i.e., first communication data stream 162A, second communicationdata stream 162B, and third communication data stream 162C (collectivelyreferred to as “communication data streams 162”, “plurality ofcommunication data streams 162”) can utilize various wireless dataprotocols or methods of transmission e.g., Bluetooth Protocols,Bluetooth Classic Protocols, Bluetooth Low-Energy Protocols, LE Audioprotocols, Asynchronous Connection-Oriented logical transport (ACL)protocols, Radio Frequency (RF) communication protocols, WiFi protocols,Near-Field Magnetic Inductance (NFMI) communications, LE AsynchronousConnection (LE ACL) logical transport protocols, or any other method oftransmission of wireless data suitable for sending and/or receivingaudio and voice data streams. In one example, the plurality ofcommunication data streams 162 can utilize at least one negotiatedisochronous data stream, e.g., a broadcast isochronous stream and/or oneor more connected isochronous streams of LE Audio protocols and may alsoutilize the LC3 audio codec. In other examples, third communication datastream 162C established between first wearable device 102A and secondwearable audio device 102B utilizes an asynchronous connection-oriented(ACL) logical transport protocol; however, it should be appreciated thatthe third communication data stream 162C can be a broadcast isochronousstream or a connected isochronous stream between first wearable device102A and second wearable device 102B. It should be appreciated that eachcommunication data stream of plurality of communication data streams 162can include a communication stream that utilizes at least one of theprotocols listed above in any conceivable combination. Additionally,each device of system 100 can be paired to at least one other devicewithin the system. In one example, first wearable device 102A and secondwearable device 102B are paired audio devices, e.g., paired trulywireless earbuds or paired speakers. As used herein, the term “paired”,along with its ordinary meaning to those with skill in the art, isintended to mean, establishing a data connection between two devicesbased on a known relationship and/or identity of the devices. Thedevices may initially exchange credentials, e.g., a Bluetooth passkey,between each other, and establish a connection between the two devicesthat share the passkey. The exchange of credentials can take place in aspecial pairing mode of the two devices to indicate ownership of bothdevices and/or the intent to pair. Once the devices are paired, they arecapable of establishing future connections based on the shared passkeyand/or the known identity of the devices. Similarly, one or more of thewearable devices 102 can be paired with peripheral device 104 to aid inestablishing the first and second communication data streams 162A-162Bdiscussed above.

During operation, a user of system 100 may engage in various actions,activities, or applications, which include or require the capture ofaudio data from the environment surrounding the user through one or moreof the first set of audio capture sensors 120 and/or one or more of thesecond set of audio capture sensors 142. As discussed above, the audiodata captured by the one or more audio capture sensors of the first orsecond sets of audio capture sensors 120,142 represents the electronicsignals obtained by the one or more audio capture sensors and includesacoustic energy generated from sources such as ambient acoustic energyfrom the surrounding environment, e.g., wind, conversations, trafficnoise, machinery, etc. The audio data can also include voice data, i.e.,voice data 164. Voice data 164 refers to the portion of acoustic energyobtained or picked up from the user's speech, e.g., vocalized acousticenergy from the user. Therefore, it should be appreciated that the audiodata includes not only ambient acoustic energy from the noises occurringin the surrounding environment of the user, but also the acoustic energygenerated by the user while speaking, i.e., voice data 164. Thus, asoutlined herein, the signals, energy levels, and average energy levelsdiscussed above can include energy corresponding to ambient acousticenergy as well as acoustic energy from the user's speech, i.e., voicedata 164. For example, first signal 122 and first energy level 124associated with the acoustic energy picked up from at least one audiocapture sensor of the first set of audio capture sensors 120, and firstaverage energy level 126 associated with the acoustic energy averagedbetween two or more audio capture sensors of the first set of audiocapture sensors 120, can include energy attributable to ambient acousticsources and human speech. Similarly, second signal 144 and second energylevel 146 associated with the acoustic energy picked up from at leastone audio capture sensor of the second set of audio capture sensors 142,and second average energy level 148 associated with the acoustic energyaveraged between two or more audio capture sensors of the second set ofaudio capture sensors 142, can include energy attributable to ambientacoustic sources and human speech.

In some applications, the ambient noise surrounding the user will not beomnidirectional or symmetrical, e.g., significantly more ambient noisemay be received proximate to the user's right ear than at the user'sleft ear. This can be caused by a number of factors including winddirected at the user's right ear (which typically results in less windpassing the user's left ear as the user's head blocks the direct windenergy). This could also be caused by the user's orientation proximateto loud objects, vehicles, machines, people, etc. In these applicationsand circumstances, the systems and methods of the present applicationare configured to compare the first signal 122 (representing the totalenergy proximate the right ear) with the second signal 144 (representingthe total energy proximate the left ear). When the comparison of thesetwo signals results in one signal having a greater energy level (i.e.,louder) than the other, the additional acoustic energy of the highersignal can be attributed to excessive ambient noise. For example, whilethe user is speaking, the first wearable device 102A (e.g., in theuser's right ear) and the second wearable device 102B (e.g., in theuser's left ear) are both equal-distant from the user's mouth. Thus, theamount of acoustic energy obtained by the audio capture sensors of thefirst set of audio capture sensors 120 of the first wearable device 102Athat is attributable to voice data 164 and the amount of acoustic energyobtained by the audio capture sensors of the second set of audio capturesensors 142 of the second wearable device 102B that is attributable tovoice data 164 are substantially equal. Thus, any difference ordisparity between the first signal 122 and the second signal 144 issolely attributable to a difference or disparity in the ambient noisedirected to the respective wearable device 102 that has the higher (orlouder) signal. In other words, the signal generated by the wearabledevice 102 with less total energy has a higher signal-to-noise ratio andthe signal generated by the wearable device 102 with more total energyhas a lower signal-to-noise ratio. Therefore, when choosing or selectingan audio capture device to be responsible for obtaining voice data 164for different applications, e.g., for voice pick up for a telephonecall, the present systems and methods choose or select the audio capturesensor for voice data pickup from based on which audio capture sensor isgenerating the lowest energy level (quietest) signal and therefore hasthe higher signal-to-noise ratio. It should be appreciated that thefirst signal 122 and the second signal 144, used in the comparisonsbelow, can be normalized, e.g., can account for any automated gaincontrol parameters or settings applied to the first signal 122 or thesecond signal 144. In other words, any automated gain control settingsapplied to first signal 122 or second signal 144 can be taken intoaccount when comparing first energy level 124 and second energy level146. Conversely, the comparisons discussed herein can also compare thefirst signal 122 and the second signal 144 to determine whether thefirst energy level 124 is greater than or less than the second energylevel 146 based on raw sensor data prior to any processing, e.g.,without taking into account any automated gain control settings appliedto the first signal 122 or second signal 144.

During operation as illustrated in FIG. 4A, the user of system 100 ispositioned such that wind approaches the user's right side, i.e.,directed toward first wearable device 102A in the user's right ear. Inthis example, first wearable device 102A is configured to obtain a firstsignal 122 associated with first energy level 124 via at least one audiocapture sensor 120 or obtain first signal 122 associated with a firstaverage energy level 126 representing the average energy level of two ormore audio capture sensors of the first set of audio capture sensors120, and store first signal 122, first energy level 124, and/or firstaverage energy level 126 in its own memory (i.e., first memory 110) orsend first signal 122 to peripheral device 104 via first comminationdata stream 162A where peripheral device 104 is configured to determinefirst energy level 124 and/or first average energy level 126. Alsowithin this example, second wearable device 102B is configured to obtaina second signal 144 associated with second energy level 146 of at leastone audio capture sensor 142 or obtain second signal 144 associated witha second average energy level 148 representing an average energy levelof two or more audio capture sensors of the second set of audio capturesensors 142, and store second signal 144, second energy level 146,and/or second average energy level 148 in its own memory (i.e., secondmemory 132) or send second signal 144 to peripheral device 104 viasecond commination data stream 162B where peripheral device 104 isconfigured to determine second energy level 146 and/or second averageenergy level 148. Upon a determination that the first signal 122 has alower energy level that the second signal 144, e.g., where the firstenergy level 124 is less than second energy level 146 (caused by, forexample, wind increasing the total energy in the audio capture sensorsof the second set of audio capture sensors 142), and thus has a highersignal-to-noise ratio, system 100 can assign the first wearable device102A the role of primary voice capture device 166. Once assigned theprimary voice capture device role 166, the first wearable device 102Acan utilize one or more audio capture sensors of the first set of audiocapture sensors 120 to obtain the voice data 164 (e.g., to obtain orpickup and record the user's speech) and process and/or send the voicedata 164 so that it may be used with one or more applications executableon peripheral device 104, e.g., for use in a telephone call.

In the alternative, as illustrated in FIG. 4B, the user of system may bepositioned such that wind approaches the user's left side, i.e.,directed toward second wearable device 102B in the user's left ear. Asdiscussed above, first signal 122 and second signal 144 are obtained bythe respective audio capture sensors of each device and are compared.Upon a determination that the first signal 122 has a higher energy levelthan the second signal 144, e.g., where the first energy level 124 isgreater than second energy level 146 (caused by, for example, windincreasing the total energy in the audio capture sensors of the firstset of audio capture sensors 120), and thus has a lower signal-to-noiseratio, system 100 can assign the second wearable device 102B the role ofprimary voice capture device 166. Second wearable device 102B, onceassigned the primary voice capture device role 166, can utilize one ormore audio capture sensors of the second set of audio capture sensors142 to obtain the voice data 164 (e.g., to obtain or pickup and recordthe user's speech) and process and/or send the voice data 164 so that itmay be used with one or more applications executable on peripheraldevice 104, e.g., for use in a telephone call.

Once selected, the primary voice capture device 166 is responsible forcompressing, encoding and sending voice data 164, obtained from therespective audio capture sensors of the first and second sets of audiocapture sensors outlined above, to peripheral device 104 for decoding,mixing, and/or use in one or more applications executed on theperipheral device 104. In other words, the primary voice capture device166 is responsible for collecting or obtaining the voice data 164 forboth wearable devices 102 and will send voice data 164 to peripheraldevice 104 for use in one or more applications. As such, should firstwearable device 102A be selected as the primary audio capture device166, second wearable device 102B can utilize third communications datastream 162C to send audio data and voice data 164 to first wearabledevice 102A for forwarding to peripheral device 104. Similarly, shouldsecond wearable device 102B be selected as the primary audio capturedevice 166, first wearable device 102A can utilize third communicationsdata stream 162C to send audio data and voice data 164 to secondwearable device 102B for forwarding to peripheral device 104.

In the event that multiple audio capture sensors are used (e.g., two ormore audio capture sensors of a particular set of audio capture sensors)to obtain voice data 164, it may be necessary to mix the voice data 164obtained by the two or more audio capture sensors prior to sending thevoice data 164 to peripheral device 104. To that end, each device ofsystem 100, i.e., first wearable device 102A, second wearable device102B and peripheral device 104, can each store and execute, on theirrespective memories and processors, a mixing algorithm 168. The mixingalgorithm 168 can be utilized to combine or mix the audio inputs of thetwo or more audio capture sensors on the selected primary voice capturedevice 166 so such that it can be outputted, sent, or utilized by theone or more applications executed on the peripheral device 104. Itshould be appreciated that the mixing algorithm can also include orutilize beamforming techniques for the inputs from the two or more audiocapture sensors to enhance the quality of the voice data 164 utilized.It should also be appreciated that, in the alternative to mixing theaudio data from two or more audio capture sensors within the circuitryof the selected primary voice capture device 166, the primary voicecapture device 166 can simply compress and send the audio data capturedby each audio capture sensor to peripheral device 104 for mixing byperipheral device 104. Metadata 170 may be utilized by the primary voicecapture device 166 (i.e., either first wearable device 102A or secondwearable device 102B) when mixing the audio data captured by the two ormore audio capture sensors of the respective device selected as theprimary voice capture device 166. For example, metadata 170 may includethe type of audio capture sensor (e.g., whether it is an internalmicrophone or external microphone) and aid in how much or how little ofa given signal is added or suppressed from the final output. In oneexample, the audio data from two or more audio capture sensors is sentto peripheral device 104 for mixing using mixing algorithm 168. In thisexample, the mixing algorithm 168 is configured to generate an output172 which can include or be influenced by metadata 170. Peripheraldevice 104 can be configured to send the output 172 to one or more ofthe wearable devices 102 and the output 172 can be used to automaticallyadjust a gain parameter 176 (discussed below), a volume level, or anactive noise reduction level used by the wearable devices 102. Theoutput 172 of the aforementioned mixing algorithm 168 can be used tomake decisions regarding the volume level or ANR settings of thewearable devices. For instance, based on the noise levels detected bythe microphone mixing algorithm 168 running on the peripheral device104, a signal could be sent to the wearable devices 102 telling it toadjust the volume level to compensate for environmental noise.Similarly, information about environmental noise could be sent as asignal back to the wearable devices 102 in order to automatically adjustANR levels. As will be discussed below, the automatic adjustment of eachof these parameters may be dependent on a switch of roles between thefirst wearable device 102A and the second wearable device 102B, e.g., aswitch in which device is the primary voice capture device 166.

It should be appreciated that, in the analysis of which wearable device102 is assigned the role of primary voice capture device 166, the audiocapture sensors used can be any conceivable combination of the audiocapture sensors listed above. For example, should first wearable device102A be selected as the primary voice capture device 166, first set ofaudio capture sensors 120 can include first audio capture sensor 120Aand second audio capture sensor 120B, where first audio capture sensor120A is an external feedforward microphone and where second audiocapture sensor 120B is an internal feedback microphone. In this example,metadata 170 of each audio capture sensor can identify that the signalproduced or obtained by first audio capture sensor 120A is an externalmicrophone and the signal produced or obtained by second audio capturesensor 120B is an internal microphone, and mixing algorithm 168 maychoose to weight the audio captured by the external feedforwardmicrophone should be weighed more heavily in the mixing process than theinternal feedback microphone to produce an output with higher fidelity.In some examples, the primary voice capture device 166 may use one audiocapture sensor from the first set of audio capture sensors 120 and oneaudio capture sensor from the second set of audio capture sensors 142.For example, should first wearable device 102A be selected as theprimary voice capture device 166, first set of audio capture sensors 120can include first audio capture sensor 120A and second audio capturesensor 120B, where first audio capture sensor 120A is an externalfeedforward microphone and where second audio capture sensor 120B is aninternal feedback microphone. Additionally, the second set of audiocapture sensors 142 of the second wearable device 102B can include afirst audio capture sensor 142A and a second audio capture sensor 142Bwhere the first audio capture sensor 142A is an external feedforwardmicrophone and where second audio capture sensor 142B is an internalfeedback microphone. In this example, as first wearable device 102A isselected as the primary audio capture device, audio data captured fromthe first audio capture sensor 142A and/or second audio capture sensor142B of the second set of audio capture sensors 142 of the secondwearable device 102B can be sent to first wearable device 102A via thirdcommunication data stream 162C so that the mixing algorithm 168 canutilize at least one audio capture sensor from the first set of audiocapture sensors 120 and at least one audio capture sensor of the secondset of audio capture sensors 142.

Additionally, to aid in the sending of audio data from two or more audiocapture sensors to another device within system 100, each device canalso utilize a compression algorithm 174. The compression algorithm usedmay be a conventional compression algorithm designed for audiocompression, e.g., low-complexity subband codec (SBC), LC3 codec,advanced audio coding (AAC), or LDAC codecs, or may be anon-conventional compression algorithm, e.g., OGG, MP3, M4A, etc. Thecompression algorithm 174 may use any of the foregoing compressioncodecs or formats when compressing and sending audio data from onedevice to another device within the system, i.e., through communicationdata streams 162A-162C.

During operation, the selection of primary voice capture device 166 orswitching the role of primary voice capture device 166 can be a dynamicprocess. For example, in real-time, the first wearable device 102A andthe second wearable device 102B can utilize one or more algorithms toperiodically, or continuously, obtain audio data captured from each ofthe audio capture sensors of the first set of audio capture sensors 120and each audio capture second of the second audio capture sensors 142and determine whether first signal 122 or second signal 144 has agreater energy level, i.e., whether first energy level 124 or secondenergy level 146 is greater. Therefore, the selection of the audiocapture sensors used to obtain voice data 164 and send voice data 164 toperipheral device for use with one or more applications is also dynamic.

In one example, during operation system 100 can dynamically perform aswitch of the primary voice capture device 155 from one wearable device102 to the other where the peripheral device 104 is aware of the switch,where the communication data streams being used utilize classicBluetooth communication protocols. For example, system 100 can beconfigured such that first communication data stream 162A betweenperipheral device 104 and first wearable device 102A and secondcommunication data stream 162B between peripheral device 104 and secondwearable device 102B are classic Bluetooth data streams. In thisexample, one of the wearable device 102 is responsible for obtainingvoice data 164 and sending it to peripheral device 104 for use with oneor more applications. Should the signal-to-noise ratio of the primaryvoice capture device 166 be too low and require a voice data 164 to beobtained from the other device, i.e., not the primary voice capturedevice. It is not desirable to obtain the voice data from the otherwearable device 102, send the voice data 164 from the other wearabledevice 102 to the primary voice capture device 166 via thirdcommunication data stream 162C, and then send the voice data toperipheral device via the first communication data stream 162A. Thetotal latency of this relay of voice data 164 is too high and wouldresult in diminishing user experience. Thus, a role switch of theprimary voice capture device 166 is needed. For example, one of thewearable devices 102, i.e., the primary voice capture device 166 isinitially responsible for receiving voice data 164 and sending the voicedata 164 to the peripheral device for use with one or more applications.Additionally, in this example, the primary voice capture device 166 canbe responsible for receiving any playback audio data to playback in bothwearable devices 102 and for sending the playback audio data to theother wearable device 102. In other words, the other wearable device 102sends and receives all audio data to the peripheral device 104 via aseparate connection with the primary voice capture device 166, e.g.,over third communication data stream 162C. In the event that the role ofprimary voice capture device 166 is switched from, for example, firstwearable device 102A to second wearable device 102B, the responsibilityof sending the voice data 164 to peripheral device 104 also switches. Inthis example, once the determination is made to switch the role ofprimary voice capture device 166 from first wearable device 102A tosecond wearable device 102B (e.g., when the second energy level 146 isless than first energy level 124), a request can be sent to theperipheral device 104 requesting that the peripheral device 104acknowledge that a role switch is about to occur. Upon receiving theresponse to the request from the peripheral device 104, the first andsecond wearable devices 102 can perform a handshake and initiate aswitch of the role of primary voice capture device 166 from firstwearable device 102A to second wearable device 102B. Thus, secondwearable device 102B becomes the new primary voice capture device 166and is responsible for relaying all audio data to and from firstwearable device 102A to peripheral device 104 and is responsible forsending voice data 164 to peripheral device 104 for use with one or moreapplications.

In another example, during operation system 100 can dynamically performa switch of the primary voice capture device role 166 from one wearabledevice 102 to the other where the peripheral device is aware of theswitch, and where the communication data streams being used utilizenegotiated isochronous data streams. For example, system 100 can beconfigured such that first communication data stream 162A betweenperipheral device 104 and first wearable device 102A and secondcommunication data stream 162B between peripheral device 104 and secondwearable device 102B are negotiated connected isochronous streams. Thus,peripheral device 104 is configured to send data streams specific toeach wearable device 102 to each wearable device 102. In other examples,there may only be one communication data stream, e.g., firstcommunication data stream 162A between the peripheral device 104 and thewearable device 102, i.e., where the first communication data stream162A is a broadcast isochronous stream. Thus, peripheral device 104 isconfigured to send a single stream of data where portions of the streamare specific to first wearable device 102A and portions of the streamare specific to wearable device 102B. In either of these examples,peripheral device 104 is initially configured to receive the voice data164 from the selected primary voice capture device 166, e.g., firstwearable device 102A. In the event that the role of primary voicecapture device 166 is switched from first wearable device 102A to secondwearable device 102B, the responsibility of sending the voice data 164from both wearable devices 102 can also switch. In this example, oncethe determination is made to switch the role of primary voice capturedevice 166 from first wearable device 102A to second wearable device102B (e.g., when the second energy level 146 is less than first energylevel 124), a request can be sent to the peripheral device 104requesting that the peripheral device 104 acknowledge that a role switchis about to occur. Upon receiving the response to the request from theperipheral device 104, the first and second wearable devices 102 canperform a handshake and initiate a switch of the role of primary voicecapture device 166 from first wearable device 102A to second wearabledevice 102B. In other words, the peripheral device 104 is configured toknow that the switch is occurring or is about to occur and will adjustits radio accordingly, e.g., peripheral radio 160, for example byadjusting the time periods when it is expecting to receive voice datapackets from the new primary voice capture device 166.

Alternatively, the dynamic selection process for selection of theprimary voice capture device 166 can be achieved without informing oralerting the peripheral device 104, e.g., without the peripheral device“knowing” a switch is about to occur or is occurring. For example, thedetermination to switch the role of primary voice capture device 166from one wearable device 102 to the other is made within or between thetwo wearable device 102 mid-stream, e.g., through the thirdcommunications data stream 162C, without providing any indication orrequest to peripheral device 104 that a switch is occurring. In thisexample, the wearable devices switch roles entirely, i.e., the firstwearable device 102A will receive and acknowledge data packets (assuminga connected isochronous stream) originally designated for secondwearable device 102B and second wearable device 102B will receive andacknowledge packets (assuming a connected isochronous stream) originallydesignated for first wearable device 102A such that the peripheraldevice 104 does not recognize that a change occurred. If singlebroadcast isochronous stream from the peripheral device 104 to wearabledevice 102 is used, each device simply receives the packets originallydesignated for the other device without either device acknowledgingreceipt of any packets.

To aid in switching roles each device may exchange access codes specificfor granting access to the specific portions of a given data stream thatare designated for each device. For example, in the event the devices ofsystem 100 utilize connected isochronous data streams, e.g., where oneor more of communication data streams 162A-162C are connectedisochronous data streams, first wearable device 102A can utilize a firstaccess code to authenticate or access and allow use of data provided byperipheral device 104 within first communication data stream 162A.Similarly, second wearable device 102B can utilize a second access codeto authenticate or access and allow use of data provided by peripheraldevice 104 within second communication data stream 162B. Prior to anyrole switch occurring between first wearable device 102A and secondwearable device 102B, these devices may exchange the first and secondaccess codes through third communication data stream 162C such thatafter the role switch occurs each device can seamlessly receive andaccess the packets of the new data stream, e.g., after the switch, firstwearable device 102A will be able to access, receive and acknowledgepackets sent within second communications data stream 162B and secondwearable device 102B will be able to access, receive and acknowledgepackets sent within first communications data stream 162A.

In one example, a gain parameters 176, a volume setting and/or an activenoise reduction (ANR) setting for each wearable device 102 can also beexchanged between first wearable device 102A and second wearable device102B prior to switching roles. Gain parameter 176 is intended to be asetting or value representing the gain applied to a particular signal,e.g., first signal 122 or second signal 144 related to voice data 164 orthe gain applied to the other audio data sent or received fromperipheral device 104, e.g., the other speaker's voice in a telephoneconversation with the user or other media inputs. By exchanging theseparameters and settings, once a role switch is effected from, e.g.,first wearable device 102A to second wearable device 102B, the newprimary voice capture device 166 (e.g. second wearable device 102B) canapply the same gain parameter 176, volume settings, and/or ANR settingsused by first wearable device 102A such that the users of the system donot perceive a change or discontinuity in voice playback or audioplayback once the switch takes effect.

FIG. 5 illustrates a flow chart corresponding to method 200 according tothe present disclosure. As shown, method 200 can include, for example:detecting a first signal 122 corresponding with a user's voice using atleast one audio capture sensor 120A of a first wearable device 102A(step 202); detecting a second signal 144 corresponding with the user'svoice using at least one audio capture sensor 142A of a second wearabledevice 102B, wherein the second wearable device 102B is wirelesslyconnected to the first wearable device 102A (step 204); determining anenergy level 124 of the first signal 122 (step 206); determining anenergy level 146 of the second signal 144 (step 208); and selecting,based at least in part on the energy level 124 of the first signal 122and the energy level 146 of the second signal 144, at least one audiocapture sensor (120A,120B) of the first wearable device 102A and/or atleast one audio capture sensor (142A,142B) of the second wearable device102B to obtain voice data 164 (step 210).

All definitions, as defined and used herein, should be understood tocontrol over dictionary definitions, definitions in documentsincorporated by reference, and/or ordinary meanings of the definedterms.

The indefinite articles “a” and “an,” as used herein in thespecification and in the claims, unless clearly indicated to thecontrary, should be understood to mean “at least one.”

The phrase “and/or,” as used herein in the specification and in theclaims, should be understood to mean “either or both” of the elements soconjoined, i.e., elements that are conjunctively present in some casesand disjunctively present in other cases. Multiple elements listed with“and/or” should be construed in the same fashion, i.e., “one or more” ofthe elements so conjoined. Other elements may optionally be presentother than the elements specifically identified by the “and/or” clause,whether related or unrelated to those elements specifically identified.

As used herein in the specification and in the claims, “or” should beunderstood to have the same meaning as “and/or” as defined above. Forexample, when separating items in a list, “or” or “and/or” shall beinterpreted as being inclusive, i.e., the inclusion of at least one, butalso including more than one, of a number or list of elements, and,optionally, additional unlisted items. Only terms clearly indicated tothe contrary, such as “only one of” or “exactly one of,” or, when usedin the claims, “consisting of,” will refer to the inclusion of exactlyone element of a number or list of elements. In general, the term “or”as used herein shall only be interpreted as indicating exclusivealternatives (i.e. “one or the other but not both”) when preceded byterms of exclusivity, such as “either,” “one of” “only one of,” or“exactly one of.”

As used herein in the specification and in the claims, the phrase “atleast one,” in reference to a list of one or more elements, should beunderstood to mean at least one element selected from any one or more ofthe elements in the list of elements, but not necessarily including atleast one of each and every element specifically listed within the listof elements and not excluding any combinations of elements in the listof elements. This definition also allows that elements may optionally bepresent other than the elements specifically identified within the listof elements to which the phrase “at least one” refers, whether relatedor unrelated to those elements specifically identified.

It should also be understood that, unless clearly indicated to thecontrary, in any methods claimed herein that include more than one stepor act, the order of the steps or acts of the method is not necessarilylimited to the order in which the steps or acts of the method arerecited.

In the claims, as well as in the specification above, all transitionalphrases such as “comprising,” “including,” “carrying,” “having,”“containing,” “involving,” “holding,” “composed of,” and the like are tobe understood to be open-ended, i.e., to mean including but not limitedto. Only the transitional phrases “consisting of” and “consistingessentially of” shall be closed or semi-closed transitional phrases,respectively.

The above-described examples of the described subject matter can beimplemented in any of numerous ways. For example, some aspects may beimplemented using hardware, software or a combination thereof. When anyaspect is implemented at least in part in software, the software codecan be executed on any suitable processor or collection of processors,whether provided in a single device or computer or distributed amongmultiple devices/computers.

The present disclosure may be implemented as a system, a method, and/ora computer program product at any possible technical detail level ofintegration. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent disclosure.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present disclosure may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some examples, electronic circuitry including, forexample, programmable logic circuitry, field-programmable gate arrays(FPGA), or programmable logic arrays (PLA) may execute the computerreadable program instructions by utilizing state information of thecomputer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to examples of thedisclosure. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

The computer readable program instructions may be provided to aprocessor of a, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks. These computer readable program instructions may also be storedin a computer readable storage medium that can direct a computer, aprogrammable data processing apparatus, and/or other devices to functionin a particular manner, such that the computer readable storage mediumhaving instructions stored therein comprises an article of manufactureincluding instructions which implement aspects of the function/actspecified in the flowchart and/or block diagram or blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousexamples of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

Other implementations are within the scope of the following claims andother claims to which the applicant may be entitled.

While various examples have been described and illustrated herein, thoseof ordinary skill in the art will readily envision a variety of othermeans and/or structures for performing the function and/or obtaining theresults and/or one or more of the advantages described herein, and eachof such variations and/or modifications is deemed to be within the scopeof the examples described herein. More generally, those skilled in theart will readily appreciate that all parameters, dimensions, materials,and configurations described herein are meant to be exemplary and thatthe actual parameters, dimensions, materials, and/or configurations willdepend upon the specific application or applications for which theteachings is/are used. Those skilled in the art will recognize, or beable to ascertain using no more than routine experimentation, manyequivalents to the specific examples described herein. It is, therefore,to be understood that the foregoing examples are presented by way ofexample only and that, within the scope of the appended claims andequivalents thereto, examples may be practiced otherwise than asspecifically described and claimed. Examples of the present disclosureare directed to each individual feature, system, article, material, kit,and/or method described herein. In addition, any combination of two ormore such features, systems, articles, materials, kits, and/or methods,if such features, systems, articles, materials, kits, and/or methods arenot mutually inconsistent, is included within the scope of the presentdisclosure.

What is claimed is:
 1. A method for selecting one or more audio capturesensors of wearable devices, the method comprising: detecting a firstsignal corresponding with a user's voice using at least one audiocapture sensor of a first wearable device; detecting a second signalcorresponding with the user's voice using at least one audio capturesensor of a second wearable device, wherein the second wearable deviceis wirelessly connected to the first wearable device; sending voice datafrom the first wearable audio device to a peripheral device; and inresponse to determining that the second signal has a highersignal-to-noise ratio than the first signal, sending further voice datafrom the second wearable audio device to the peripheral device.
 2. Themethod of claim 1, wherein the determining that the second signal has ahigher signal-to-noise ratio than the first signal is performed by thefirst wearable device.
 3. The method of claim 1, wherein the determiningthat the second signal has a higher signal-to-noise ratio than the firstsignal is performed by the second wearable device.
 4. The method ofclaim 1, wherein the determining that the second signal has a highersignal-to-noise ratio than the first signal is performed by theperipheral device.
 5. The method of claim 1, wherein the first wearabledevice is intended to fit around, on, in, or near a first ear of theuser, and the second wearable device is intended to fit around, on, in,or near a second ear of the user.
 6. The method of claim 1, furthercomprising, in response to determining that the second signal has ahigher signal-to-noise ratio than the first signal, causing a roleswitch from the first wearable device being a primary device to thesecond wearable device being the primary device.
 7. The method of claim1, wherein the determining that the second signal has a highersignal-to-noise ratio includes comparing energy levels of the first andsecond signals to determine that the first signal has higher energylevels than the second signal.
 8. The method of claim 8, wherein thefirst signal has higher energy levels than the second signal due to ahigher wind presence in the first signal than the second signal.
 9. Themethod of claim 1, wherein when the second wearable audio device sendsthe further voice data to the peripheral device, the first wearabledevice no longer sends the voice data to the peripheral device.
 10. Themethod of claim 1, wherein the at least one audio capture sensor of thefirst wearable device and the at least one audio capture sensor of thesecond wearable device are selected from at least one of an exteriormicrophone, an interior microphone, a feedback microphone that is alsoused for acoustic noise reduction (ANR) purposes, a feedforwardmicrophone that is also used for ANR purposes, or an accelerometer. 11.A system comprising: a first wearable device that includes at least oneaudio capture sensor configured to detect a first signal correspondingwith a user's voice; a second wearable device that includes at least oneaudio capture sensor configured to detect a second signal correspondingwith the user's voice, the second wearable device wirelessly connectedto the first wearable device; and non-transitory computer readableinstructions executed by at least one processor of the first wearabledevice and/or the second wearable device, the instructions configured tocause the first wearable device to send voice data to a peripheraldevice, and, in response to determining that the second signal has ahigher signal-to-noise ratio than the first signal, cause the secondwearable device to send further voice data to the peripheral device. 12.The system of claim 11, wherein the determining that the second signalhas a higher signal-to-noise ratio than the first signal is performed bythe at least one processor of the first wearable device.
 13. The systemof claim 11, wherein the determining that the second signal has a highersignal-to-noise ratio than the first signal is performed by the at leastone processor of the second wearable device.
 14. The system of claim 11,wherein the determining that the second signal has a highersignal-to-noise ratio than the first signal is performed by at least oneprocessor of the first wearable device and at least one processor of thesecond wearable device.
 15. The system of claim 11, wherein the firstwearable device is intended to fit around, on, in, or near a first earof the user, and the second wearable device is intended to fit around,on, in, or near a second ear of the user.
 16. The system of claim 11,wherein the instructions are further configured to, in response todetermining that the second signal has a higher signal-to-noise ratiothan the first signal, cause a role switch from the first wearabledevice being a primary device to the second wearable device being theprimary device.
 17. The system of claim 11, wherein the determining thatthe second signal has a higher signal-to-noise ratio includes comparingenergy levels of the first and second signals to determine that thefirst signal has higher energy levels than the second signal.
 18. Thesystem of claim 18, wherein the first signal has higher energy levelsthan the second signal due to a higher wind presence in the first signalthan the second signal.
 19. The system of claim 11, wherein when thesecond wearable audio device sends the further voice data to theperipheral device, the instructions are further configured to stop thefirst wearable device from sending the voice data to the peripheraldevice.
 20. The system of claim 11, wherein the at least one audiocapture sensor of the first wearable device and the at least one audiocapture sensor of the second wearable device are selected from at leastone of an exterior microphone, an interior microphone, a feedbackmicrophone that is also used for acoustic noise reduction (ANR)purposes, a feedforward microphone that is also used for ANR purposes,or an accelerometer.